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Abstract 

In this paper, the performance limits and the computational complexity of the lattice sequential 
decoder are analyzed for the unconstrained additive white Gaussian noise channel. The performance 
analysis available in the literature for such a channel has been studied only under the use of the minimum 
Euclidean distance decoder that is commonly referred to as the lattice decoder. Lattice decoders based 
on solutions to the NP-hard closest vector problem are very complex to implement, and the search for 
low complexity receivers for the detection of lattice codes is considered a challenging problem. However, 
the low computational complexity advantage that sequential decoding promises, makes it an alternative 
solution to the lattice decoder. In this work, we characterize the performance and complexity tradeoff 
via the error exponent and the decoding complexity, respectively, of such a decoder as a function of 
the decoding parameter — the bias term. For the above channel, we derive the cut-off volume-to-noise 
ratio that is required to achieve a good error performance with low decoding complexity. 

I. Introduction 

The theory of lattices — a mathematical approach for representing infinite discrete points 
in Euclidean space Q, has become a powerful tool to analyze many point-to-point digital and 
wireless communication systems, particularly, communication systems that can be well-described 
by the linear Gaussian vector channel model. This is mainly due to the three facts about 
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channel codes constructed using lattices: they have simple structure, their ability to achieve the 
fundamental limits (the capacity) of the channel, and most importantly, they can be decoded using 
efficient decoders called lattice decoders 0. Many researchers have studied the information- 
theoretic limits of lattice coding and decoding schemes for the linear Gaussian vector channel 
model ED-0. 

Poltyrev [3J studied the problem of coding for the unconstrained additive white Gaussian 
noise (AWGN) channel where the channel input being an infinite lattice. In his setting, the 
notion of capacity becomes meaning less as infinite rates of transmission are possible. Therefore, 
another significant measurement was defined that characterizes the performance limits of such 
coding scheme when decoded using lattice decoders — the normalized density of the lattice or 
equivalently the information density rate of the lattice. 

Based on a random lattice coding technique, Poltyrev showed that, using lattice decoding, the 
average probability of error can be upper bounded as 

Pe,M < e~ mE ^\ (1) 

where m is the dimension of the lattice code, and E p (p, c ) is called the Poltyrev error exponent 
and is shown to be a non-zero, monotonically decreasing, positive function for all \i c > 1. 
The parameter \i c , which is called the volume-to-noise ratio (will be defined in the sequel), is 
a quantity that is related to the density of the lattice. Hence, [L c — 1 has the significance of 
capacity. 

In contrast to Poltyrev, where random coding has been used to prove the above result, Loeliger 
JH showed that the above upper bound can be achieved using ensembles of linear lattices — 
constructed using linear codes over the ring of p-prime integer numbers, i.e., Z p , which is usually 
referred to as Construction A 0]. An important aspect of both Poltyrev 's and Loeliger' s proofs 
is based on an important theorem in number theory that is referred to as Minkowski-Hlawka 
theorem M, M- 

It is clear from the above bound that large lattice codes would be required to approach 
capacity and therefore more practical decoding methods would be needed. It is well-known that 
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lattice decoders that are implemented using sphere decoding algorithms^ can be considered as a 
search in a tree (see IfTOll . ifTTI and references therein). Generally speaking, a sphere decoding 
algorithm explores the tree of all possible lattice points and uses a path metric in order to discard 
paths corresponding to points outside the search sphere. Unfortunately, sphere decoding suffers 
from high computational complexity for low-to-moderate volume-to-noise ratios^] and for large 
signal dimensionality in which low error probability is to be expected IfTOll . As an alternative 
to sphere decoding algorithms, sequential decoders comprise a set of efficient and powerful 
decoding techniques able to perform the tree search. These decoders can achieve near-optimal 
performance without suffering the complexity of the sphere decoder for coding rates not too 
close to capacity 1121. lfT3l 

The stack algorithm is a well-known algorithm that is used to describe the operation of the 
sequential decoder |fT3l . The algorithm was originally constructed as an alternative approach to 
the maximum-likelihood (ML) decoder for detecting convolutional codes transmitted via discrete 
memory less channels. It has been shown in [TT2l that as long as we operate below the cutoff 
rate, the decoder can achieve near-ML performance with low decoding complexity. 

For the lattice coded/decoded linear Gaussian vector channel model, there is a small body of 
work that discusses the performance and complexity tradeoff achieved by sequential decoding 
algorithms. Initial work on this topic was done by Tarokh et. al. lfl"5l where sequential decoding 
is used to decode lattice codes with finite trellis diagram. Shalvi et. al. in [fT6l has considered 
the use of sequential decoders to decode convolutional lattice codes. These power-limited (finite) 
lattice codes are generated using lattices combined with special lattice shaping techniques. The 

'Sphere decoding algorithms were originally implemented to decode signals transmitted via wireless fading channels 1111 . 
particularly for the quasi-static multiple-input multiple-output wireless channels as an attempt to reduce the high computational 
complexity of the optimal maximum-likelihood decoder (see 1101 ). The latter channel maybe described by the linear Gaussian 
vector channel model which allows the use of lattice coding, and lattice decoding to analyze the performance limits of such 
systems. 

2 The notion of "signal-to-noise ratio" is usually used for power-constrained channels where only a finite number of codewords 
or signals can be be transmitted. Here, for infinite lattice codes, the notion of volume-to-noise ratio is used instead which will 
be introduced in the sequel. 

3 The work in [14j considered the application of lattice sequential decoders to various systems that can be described by the 
linear Gaussian vector channel model, such as the slowly-fading multiple-input multiple-output wireless channel, and the inter- 
symbol interference channel. In this work, it has been shown that near-sphere decoding performance can be achieved without 
suffering the high decoding complexity of the sphere decoder. 
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convolutional structure of such codes allows the use of the sequential decoders to achieve high 
data rates with low decoding complexity (this was mainly shown via simulation). However, all 
previous works lack of a thorough theoretical analysis that can describe the systematic approach 
for tradeoff performance, complexity, and rate (or lattice density) achieved by sequential decoding 
of infinite lattice codes. 

The contribution of this paper can be classified into two classes: the performance analysis of 
the lattice sequential decoder in terms of the achievable error exponent, and the computational 
complexity of the decoder in terms of the "cut-off" volume-to-noise ratio via the complexity 
tail distribution. Those are derived as a function of the decoding parameter - the bias term. In 
order to fully characterize the performance of the decoder, we determine for the first time the 
error exponent achievable by lattice coding and sequential decoding applied to the unconstrained 
AWGN channel. We derive the error exponent as a function of the bias term which is critical for 
controlling the amount of computations required at the decoding stage. Achieving low decoding 
complexity requires increasing the value of the bias term. However, this is done at the expense 
of increasing erroneous detection. In this work, we follow the footsteps of Poltyrev and use the 
same definition of capacity for such a channel. We make use of lattice codes drawn from the 
ensemble of linear lattices, i.e., Loeliger construction JH. 

We analyze in details the computational complexity tail distribution of the lattice sequential 
decoder. We show that there exists a cut-off volume-to-noise ratio that yields low decoding 
complexity which is also an increasing function of the bias term. We show that achieving low 
decoding complexity with good error performance comes at the expense of increasing the cut- 
off volume-to-noise ratio. Hence, lattice sequential decoders provide a systematic approach for 
tradeoff performance, complexity, and lattice density. 

In contrast to most work in sequential decoding algorithms where the bias term is usually 
optimized to achieve a good performance-complexity tradeoff, we allow the bias term to vary 
freely and study the effect of this variation on the performance-complexity tradeoffs achieved 
by such decoders. 

Through out the paper, we use the following notation. The superscript T denotes transpose. 
For a bounded region 1Z C IR m , V(1Z) denotes the volume of 1Z. We denote «S m (r) by the 
m-dimensional hypersphere of radius r with V(S m (r)) = (nr 2 )™/ 2 /T(m/2 + 1), where T(n) = 
J °° x m ~ 1 e~ x dx, is the Gamma function. Bold lowercase letters o denote vectors, whose Z 2 -norm 
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is denoted by ||a||, and bold uppercase letters A denote matrices, where I rn denotes the m x m 
identity matrix. The notation v ~ ft/(p,,K) indicates that v is a real Gaussian random vector 
with mean and covariance matrix K, and £{■} represents the statistical average. 

II. Coding Without Restriction for the AWGN Channel 
A. Lattice Properties 

A lattice is a discrete pointset A in a Euclidean space IR m that is closed under vector addition, 
i.e., any translate A + x by a lattice point x G A is just A again. Let {gi,g2, • • • ,g m } be a set 
of linearly independent vectors in IR m . The set A of all linear combinations x = z±gi + Z2Q2 + 
■ ■ ■ + z m g m with integer coefficients z t is a lattice, i.e., 

\ = {x = Gz:zd Z m }, 

where G = \gi,g2, • • • ,9m] is an m x m full-rank generator matrix. Thus, any lattice A in IR m 
can be seen as a linear transformation of the integer lattice Z m . 

Some properties associated with the lattice A are of great importance for our analysis: 

• The nearest neighbor quantizer Q(-) associated with A is defined by 

Qa(x) = argmin II A — x\\. 
AeA 

• The Voronoi cell V(A) that corresponds to the lattice point A G A is the set of points in 
R m closest to x, i.e., 

V(A) = {xe W m : Q A (x) = A}. 

Voronoi cells associated with each lattice point A G A are congruent and therefore can be 
considered as a shift of V(0) by A. 

• The volume of the Voronoi cell is given by 

V(G) = Vol(V(0)) = Vdet (G T G), 

with the property that V(aG) = a m V(G) for any a > 0. 

• The covering radius r cov (A) is the radius of the smallest sphere centered at the origin that 
contains V(0). The effective radius r e ff(A) is the radius of the sphere with volume equal to 
V(G). The packing radius r pack (A) is the radius of the largest sphere centered at the origin 
inside the Voronoi cell V(0) (see Figure [T]). 
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Fig. 1. The packing radius, the effective radius, and the covering radius of the hexagonal lattice. 

• A sequence of lattices {A m } of increasing dimension is good for covering j6l if r cov (A m ) — > r cS (A m ). 

• Minkowski-Hlawka Theorem [8J: Let / : IR m — > R be a Riemann integrable function of 
bounded support (i.e., f(x) = if exceeds some bound). For any S > 0, there exist 
ensembles {A} of lattices with volume V(G) and dimension m such that 



where the expectation £\ is taken over the ensemble of random lattices, A* = A\{0}, and 
5 — > as m — > oo. The above important theorem is sometimes regarded as a pre-Shannon 
result in information theory. In fact, the Mikowski-Hlawka theorem was originally used for 
packing lattices to solve the well-known sphere-packing problem J9]|. 

B. Poltyrev Error Exponent 

Suppose that an m-dimensional lattice point x = G c z E A c is to be transmitted through the 
unconstrained AWGN channel, where A c is an infinite lattice code with volume V c = V(G C ), 
that is drawn from the ensemble of linear lattices using Loeliger construction (see H for more 
details about the construction). The received vector (output of the channel) in this case can be 
mathematically expressed as 




(2) 



y = x i w. 



(3) 
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where w ~ J\f(Q,a 2 I m ). Due to the unconstrained power condition on the lattice codewords 
(points), the optimum receiver that minimizes the probability of decoding error can be expressed 
as 

z = arg min \\y — G c z\\ 2 , (4) 

which corresponds to searching over the whole lattice A c to find the closest point to the received 
vector y. This is referred to as lattice decoding. 

As mentioned in the introduction, Poltyrev studied the problem of coding for the unconstrained 
AWGN channel with the input alphabet being the whole space IR m . Since infinite power is 
possible, the notion of capacity becomes meaningless. Instead, the decoding error probability is 
measured against the normalized per dimension volume-to-noise ratio (VNR), fx c , defined by 

4 V{G c f/ m = 

1-nea 2 27rea 2 ' 

where (Vc /2ire) ■ m represents the asymptotic (in dimension m) squared radius of a sphere 
of volume V c . 

Poltyrev showed that the average probability of error (averaged over the ensemble of linear 
lattice codes A c ) is upper bounded by ([[]) where 

\ [(ji e - 1) -log// c ] , 1 < fi c < 2; 

§kg(x)' 2</i c <4; (6) 



^ c /8, n c > 4. 

From the above analysis, one can notice that fi c can be interpreted as the ratio of the squared 
radius of a spherical Voronoi cell to the variance of the noise. For small /x c , i.e., /x c < 1, the 
spherical Voronoi cell has radius less than the standard deviation of the noise. In this case, 
reliable communication is not possible as error is highly likely to occur. As such, // c = 1 has 
the significance of capacity. 

Interestingly, Poltyrev showed that if only a finite number of lattice points are to be trans- 
mitted as codewords with finite power constraint and transmission rate R, then rates R up to 
l/21og(SNR) is achievable, where SNR > here represents the average signal-to-noise ratio 
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of the channeQ For high SNRs (i.e., for SNR > 1), l/21og(SNR) represents the capacity of 
the AWGN channel, denoted by C. Therefore, the same error probability bound given in ([T]) 
and ([6]) can be used (asymptotically) to characterize the performance of the power-limited lattice 
coded/decoded AWGN channel by letting (at high SNR) /i c = 2 2 ^°~ R \ 

Unfortunately, lattice decoders (usually implemented using sphere decoding algorithms) suffer 
from high computational complexity for low-to-moderate SNR and for large signal dimensionality 
in which low error probability is to be expected. As an alternative to lattice decoders, lattice 
sequential decoders comprise a set of efficient and powerful decoding techniques that can achieve 
near-optimal performance without suffering the complexity of the lattice decoder for coding rates 
not too close to capacity C. In fact, it is well-known that sequential decoders can work well 
(with low decoding complexity) for rates below the cut-off rate -R which is only a factor of 4/e 
(1.68 dB) away from capacity C at the high-SNR regime ifTTl . Therefore, for the unconstrained 
AWGN channel, /x c = 4/e has the significance of the cut-off rate. Here, we call this the cut-off 
VNR, denoted by // . 

III. The Stack Sequential Decoder 

In this section, we briefly introduce the operation of the stack algorithm. This algorithm 
is an efficient tree search algorithm that attempts to find a "best fit" with the received noisy 
signal. Before we proceed with the description of such an algorithm, we shall discuss the metric 
measure for sequential decoding of lattice codes. It is basically based on the path metric defined 
for conventional sequential decoders which is given by lfi~4ll 

( ^%{z\))M \U{z\))' 

v m: 

where 1-L{z\) is the hypothesis that z\ form the first k symbols of the transmitted information 
sequence, and /(■) is the usual probability density function. 

Recently, it has been shown that the search for the closest lattice point problem which 
corresponds to Q can be well performed using sequential decoders based on the stack algorithm 

4 This can be simply done by intersecting the lattice code A c (possibly shifted by a vector no) with a shaping region TZ 
(a sphere or a Voronoi cell of another lattice), i.e., C = (A c + uo) PI TZ. In this case, the transmission rate is given by 
R = i log 2 [V r (7?.)/V r c ], where V(7Z) is the volume of the shaping region. If we define mP x — X^gc ll 35 !! 2 to ^ e me 
average transmitted power, then one can show that V(1Z) is asymptotically (as m — > oc ) given by {2neP x ) m/2 . For reliable 
communication, we must have V c > (2nea 2 ) m ^ 2 . Therefore, rates R up to | log(P x / a 2 ) — | log(SNR) is achievable. 



= ( ^/Ar 1 1;; ) > (7) 
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031- For our channel model, the path metric given by ([7]) can be shown to be simplified to (see 
Appendix A in 031) 

^z k )=bk-\\y' k -R kk z k \\ 2 , (8) 

where z\ = [zk, • • • ,z 2 ,zi] T denotes the last k components of the integer vector z, Rkk is the 
lower k x k matrix of R that corresponds to the QR decomposition of the code matrix G c = QR, 
y' = Q T y, and b is the bias term. 

The stack sequential decoder is an efficient tree search algorithm that attempts to find a "best 
fit" with the received noisy signal. As in the conventional stack decoder 021, to determine a 
best fit (path), values are assigned to each node on the tree. This value is called the metric which 
is given by ([8]). As the decoder searches nodes, an ordered list of previously examined paths 
of different lengths is kept in storage. Each stack entry contains a path along with its metric. 
Each decoding step consists of extending the top (best) path in the stack. The determination of 
the best and next best nodes is simplified in the closest lattice point search problem by using 
the Schnnor-Euchner enumeration 01)1 which generates nodes with metrics in ascending order 
given any node z\. The decoding algorithm terminates when the top path in the stack reaches 
the end of the tree (refer to 031 for more details about the algorithm). 

The main role of the bias term b used in the algorithm is to control the amount of computations 
performed by the decoder. In this work, we define the computational complexity of the lattice 
sequential decoder as the total number of nodes visited by the decoder during the search. Also, 
the bias term is responsible for the excellent performance-rate-complexity tradeoff achieved by 
such a decoding scheme. The role that the bias parameter plays will be discussed in details in 
the subsequent sections. 

For the stack algorithm, in order to help distinguish the correct path from all incorrect paths, the 
value of b should be chosen carefully such that the average metric along the correct path increases, 
while decreases, on average, along the incorrect path. Assuming the decoder is following the 
correct path, the average metric at level k is given by 

S{fi(zi)\ z\ correct path } = (6 - a 2 )k, 

which is positive for b > a 2 . 
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IV. Performance Analysis: An Upper Bound 

As mentioned at the introduction, there has been no analysis devoted to sequential decoding 
applied to the lattice coded unconstrained AWGN channel. In this section, we analyze the per- 
formance limits of the stack sequential decoder when lattice coding is applied at the transmitter. 
We consider the unconstrained AWGN channel as defined by Poltyrev 0. Finding the exact 
error performance of such a decoder seems to be difficult. Therefore, we attempt to derive an 
upper bound on the sequential decoding error probability. 

Define P e (b) as the probability that the sequential decoder makes an erroneous detection at 
a bias value b (defined in ([8])). Now, due to lattice symmetry, one can assume that the all-zero 
lattice point is transmitted. For a given lattice A c , we have 

P e {b\A c ) = Pr j (J {0 was decoded as x} J < Pr j (J {/i(z) > // min } 
\xeA* c J \zez m \{0} 

< Pr j [J {||x|| 2 - 2x T w < bm} 



Pr M) < 2x J w > \\x\ 



2 



bm 
\\x\\ 2 



(c) / i i f t ,,.■>/ bm 
< Pr <2xw> \\x\ 

\xeA* c L 



d 2 

mm 



(9) 

where (a) is due to the fact that in general, /i min = min{0,6 — ||t/;^|| 2 ,26 — ||w 2 || 2 , . . . ,bm — 
||w/™|| 2 } is the minimum metric that corresponds to the transmitted path, jx{z) > // m in is just a 
necessary condition for x = Gz to be decoded by the stack decoder, (b) follows by noticing that 
— (Ai m in + ||w|| 2 ) < 0, and (c) follows from the fact that ||x|| > min^A* \\x\\ = d min (the shortest 
vector in the lattice). 

It is clear from the above analysis that the sequential decoder approaches the performance of 
the lattice decoder as b — > 0. Now, as m — > oo, for a well-constructed lattice code ensemblejf] it 
is well-known that 



72 \r 2 l m 

fi c a' 



^min 2 



m 2ne 

5 Codes which are constructed using lattices that satisfy the Minkowski-Hlawka theorem (see |3|-|6| for more details). 
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Therefore, the probability of decoding error can by (asymptotically) upper bounded by 

b/a 2 ' 



Pe(b\A c ) < Pr j [J !.2x T w > \\x\\ 2 (l 
\xeA* < ^ 



/'c 




(10) 



Pr (J {2x T w > \\x\\ 2 } 



where 



W = | 1 | w 

He 



-1 



is a zero-mean Gaussian random vector with elements that are independent, identically, dis- 
tributed random variables with variance a 2 = (1 — / Ufe//i c )~ 2 cr 2 , and fib = bjo 2 is defined to be 
the normalized bias with respect to the noise variance. It must be noted that the above bound 
is only valid for all values of ^ such that 1 — /i fe //i c > 0, or equivalently for all values of 
< fi b < fi c . 

Interestingly, the upper bound ( [10] ) corresponds to the probability of decoding error of a 
received signal y = x + w decoded using the conventional lattice decoder. Therefore, one may 
observe that the sub-optimality of the sequential decoder can be viewed as a source of channel 
noise amplification. 

Following the footsteps of Poltyrev, the average probability of error (averaged over the en- 
semble of linear lattice codes A c ) can be upper bounded by 

T e {b) = £ Ac {P e {b\A c }) < e ~ mE ^\ (11) 

where 

o, fic < i; 

- 1) - log/i c ] , 1 < fic < 2; 
§l°g(^Y 2</i c <4; 
/i c /8, fl c > 4. 

^vjM^i^^.^ (13) 

Hence, for sufficiently large m, there exists at least a lattice A* in the ensemble with error 
probability satisfying 

P e (b,A* c ) < e" mEM . (14) 



E b (fj, c ) = E p (jl c 



(12) 



where 
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Now, the following important remarks can be made about the above result: 
• Fixed Bias: In this case, the bias term b is fixed and chosen independent of the VNR \i c . 
Note that as /x c gets large (// c ^> 1) , one may approximate \x c in ( [T3] ) as 

p, c = \l c ( 1 - — ] /i c ( 1 - 2— J =// c - 2/i 6 . (15) 

Therefore, the above analysis indicates that fixing the bias term causes a right-shift to the 
error probability curve (i.e., a reduction in the coding gain). This can be realized from the 
value of the error exponent for large /i c which is given by 

EM = | = f - f = EM - f , (16) 

where E p (ijl c ) is the Poltyrev error exponent achieved by the lattice decoder which is defined 
in @. Substituting ([16]) into ([14]) we get 

P e (6) < e~ mEb ^ = e - " 1 ^^) - ^/ 4 ] = ae~ mEp ^ (17) 

where a = e mw,//4 . The constant a sheds some lights on the behavior of the error probability 
of the sequential decoder for fixed bias term. Increasing the bias term results in performance 
reduction compared to the one achieved by the lattice decoder. This reduction is represented 
by a right-shift to the error probability curve^j as will be shown in the sequel. 



Variable Bias: Now, let the normalized bias /i b to scale linearly with the VNR fi c as //& 
(1 — V5)j2 c where < 5 < 1, then the error exponent in this case can be expressed as 



Eb(fJ>c 



0. 

^log 
6fi c /l 



-l)-log5/i c ], l/5<fi c <2/5; 
eSfx c 



(18) 



2/5 <fi c < 4/5; 
> 4/5. 

It is clear from the above analysis that if 5 = 1 = 0) then the performance of the 
sequential decoder approaches the performance of the lattice decoder. On the other extreme, 

6 It must be noted that, although the bound Jj^J shows that the shift is a = e mMt " /4 , the exact amount of right-shift is less 
than a as will be shown by the simulation results in Section VI. 
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Fig. 2. The achievable error exponent of the lattice sequential decoder when the normalized bias term fit, = (1 — \f5)n c for 
8 = 0.5,0.75, and 1. 

if 5 = (fib = He) then reliable communication may not be possible under lattice sequential 
decoding. Fig. 2 shows the error exponent achieved by the lattice sequential decoder for the 
case of the variable bias term described above. It is clear from Fig. 2 that for high VNR /i c , 
the affect of varying 5 occurs as a change in the slope of the error exponent curve, where 
at high VNR we have E b {ji c ) = 5fx c /8. Moreover, the achievable VNR under sequential 
decoding with normalized bias = (1 — v / 5)/i c is given by 1/5. Therefore, for 5 ^ 1, 
reliable communication may not be possible at VNR close to capacity (/i c = 1). 
Next, we consider the computational complexity analysis of the sequential decoder for the 
unconstrained AWGN channel. As was discussed above, increasing the bias term increases the 
decoding error probability as well. However, the loss in the error performance achieved by any 
sub-optimal decoder is usually compensated by some improvements in the decoding complexity. 
This fact will be demonstrated next. 
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V. The "cut-off" Volume-to-Noise Ratio 



The main use of the sequential decoder is to achieve a low decoding complexity compared to 
the very complex lattice decoder. As in conventional sequential decoder, one need to back-off 
from capacity to achieve such improvements. For convolutional codes detected using sequential 
decoder, a cut-off rate has been defined for such decoding scheme. The cut-off rate R is the rate 
for which the transmitter should not exceed if one needs to expect a low decoding complexity. 
If R > Ro, then the complexity of the sequential decoder increases exponentially with the 
constraint length of the code [[12ll . For the power-constrained AWGN channel, the cut-off rate 
Ro is 4/e (1.68 dB) away from capacity at high SNR (see [TTTIl and references therein). 

For the lattice sequential decoder, the analysis of the computational complexity is considered 
a difficult problem. As a start to solve this problem for the the unconstrained AWGN channel, 
we would like to find a lower bound on the "cut-off" VNR /i — the value of \x c for which 
low decoding complexity is possible. Such lower bound appears to be analytically tractable by 
bounding the average decoding complexity, which is defined here as the total number of nodes 
in the tree that have been visited by the decoder during the search. 

A. Average Computational Complexity: A Lower Bound 

First, one should note that all nodes in the tree that are visited by the sequential decoder prior 
decoding the message must have partial path metrics n(z\) that exceed the minimum metric 
yLi m i n that corresponds to the decoded path. Let 4>{z\) be the indicator function defined by 



0, otherwise, 

and let jV(A c ) be a random variable that denotes the total number of visited nodes during the 
search for a given lattice code A c . Then, A/"(A C ) can be expressed as 



Similar to the performance analysis, due to lattice symmetry, we assume that the all- zero lattice 
point was transmitted. It is clear that 




(19) 



m 




(20) 



A/"(A C ) > 
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Now, since ||wi|| 2 > for any 1 < k < m, then 

/i min = min{0,6-||wi|| 2 ,26-||w 2 || 2 ,. . . , mb- ||tu™|| 2 } < min{0, 6-0, 26-0, •• • ,m6-0} = 0. 

In this case, the average number of computations (averaged over the statistics of the noise) 
can be upper bounded as 

£ w {Af(Ac)} > sj (21) 
^ xeA c ' 

where 

{1, if \\w — x\\ 2 < bk; 
0, otherwise. 

We can further lower bound pTj ) as follows: 

( } r e -IMI 2 /2°- 2 

£ w \22^( x )\ = 22 fww^ fo) dw 

e -\\M?l^ 2 

(27T(J 2 ) m /2 



r e -\\H 2 /^ 2 



xeA c 

~r 2 cov /2a 2 



■|2<- r ,2 



6 LUV / _ ^ I 



ceAc ^eV(O) 

e -r c 2 ov /2a 2 



^^^(V^)). ( 22 ) 

The above lower bound is valid for any lattice code A c . Assuming the use of a well-constructed 
lattice (e.g., Loeliger ensemble [4J), we have that for large m (i.e., asm4 oo), r 2 ov — > r 2 ff = 
(y^ m /2ixe)m = jj, c a 2 m and V(S m (y/bk)) — >■ (2Ttebk/m) m / 2 . Therefore, asymptotically, the 
average number of computations can be lower bound as 

p-iic.mil 

MA/"(A c)} > J^y^i 2 ^ 2 

> . (23) 



Thus, at // 6 e 1_Mc > 1, i.e., at VNR /i c < log e (/z&e), the average number of computations per- 
formed by the sequential decoder must go to infinity exponentially as m is increased. Therefore, 
in order to ensure a very low decoding complexity as the bias b increases, the cut-off VNR /x 
should be chosen to satisfy the following lower bound 

/i >l+log e (/i 6 ). (24) 
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As argued in section III, a low decoding complexity is to be expected as long as fib = b/a 2 > 1. 
Since [iq > I +log e (//&), the cut-off VNR is strictly greater than 1, where /x c = 1 is the capacity 
of the unconstrained AWGN channel. Therefore, one has to stay away from capacity if low 
decoding complexity is to be expected. The above lower bound on fi can be further improved 
by considering the decoding complexity tail distribution as will be shown in the sequel. 



B. An Upper Bound on The Complexity Distribution 

Although sequential decoding algorithms are simple to describe, the analysis of the decoder's 
computational complexity is considered difficult. This is due to the fact that the amount of 
computations performed by the decoder attempting to decode a message is random. Therefore, 
the complexity of such a decoder is usually analyzed through its computational distribution 
defined by Pr(A/"(A c ) > L), where L is the distribution parameter. In this section we would like 
to find an upper bound on the decoding complexity distribution and to find the values of fio that 
lead to low decoding complexity. Consider now the following analysis: 

First, the complexity tail distribution can be upper bounded as 

Pr(A/"(A c ) > L) < Pr(A/"(A c ) > L, \\w\\ 2 < a 2 m) + Pr(\\w\\ 2 > a 2 m), (25) 

where the above upper bound is derived using the well-known separation of the typical noise 
events from the non-typical ones [fT8l . Next, we would like to upper bound the second term in 



the RHS of ([25]). 

Given ||w|| 2 < a 2 m, and by noticing that — (yU m i n + ||w|| 2 ) < 0, we obtain 

M) < £ *'(*}), (26) 



where <b{z\) is the indicator function defined in (19), and 



1, if lit//? -R kk z\f < bk + a 2 m; 
(-,;'): { - (27) 

0, otherwise. 



Now, let 

S k , if \\w'-Rz\\ 2 < 6m- //„ 



0, otherwise, 
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where 



S k = <£'(*i), ( 2g ) 



then it can be shown that 

m m 

^(a c ) < E E <EE 

fc=l zeZ m fc=l a;GA c 

where 

Su, if \\x\\ 2 — 2(x) T w < bm: 



» 



'it, 

0, otherwise. 



Interestingly, the sum that appears in ( [28] ) represents the number of partial integer lattice 
points z\ E 7j k that are located inside a sphere of squared radius bk + o 2 m centered at the 
received signal (y = w in our case). One can approximate S k by the ratio of the volume of 
the m-dimensional sphere of squared radius bk + a 2 m to the volume of the Voronoi cell of the 
lattice as (see flU for more details) 

^ (167r) fc / 2 [bk + o*m]W 

k ~ r(k/2 + i)(det(R[ k R kk y/*y 

For a given lattice A c , we have 

Pr(A/"(A c ) > L, ||w|| 2 < a 2 m\A c ) < Pr(A/"(A c ) >L-m, \\w\\ 2 < a 2 m\A c ) 

< «^F 1 A) i fori>m> < 30 > 

L — m 

where the last inequality follows from using Markov inequality, and Af(A c ) is defined as 

m 

#(a«) = E E 

k=i 2jez fc \{0} 

since we have assumed that the all-zero lattice point was transmitted. 

The conditional average of Af(A c ) with respect to the noise can be further upper bounded as 

in 

£ W {M(A C )\A C , \\w\\ 2 < a 2 m} < S k E Pr (ll x H 2 ~ 2 ( X V W < M- (31) 

k=l ieA* 

Therefore, we have 

Pr(AA(A c ) > L, \\w\\ 2 < a 2 m\A c ) < ^ k=1 k V Pr(2(x) T w > \\x\\ 2 - bm). (32) 

L — m t—f 
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Now, for L = m + Ylk=i we have that 

Pr(W(A c ) > L|A C ) < Pr(2(x) T w > \\x\\ 2 - bm) + Pr(||w|| 2 > a 2 m). (33) 

Using Chernoff bound, 

{ e -\\x\\ 2 /8* 2 e bm/4*2 U X U2 > bm . 
(34) 
1, ||x|| 2 < bm, 

Let 

0, llxll 2 > bm: 



1, \\x\\ < bm, 



and 

e-W 2 /^V^, ||a:|| 2 > bm; 

9{x) ={ 

10, ||x|| 2 <6m. 

Therefore, one may upper bound the first term in the RHS of ([33]) as 



Pr(2(x) T w > \\x\\ 2 -bm)<J2 /(*) + E ^ (35) 



By taking the expectation of (33) over the ensemble average of random lattices, we obtain 



Pr(A/"(A c ) > L) = £ Ac {Pr(A/-(A c ) > L|A C )} 



< 



- I f(x) dx + ^- I g(x) dx + Pr(||w|| 2 > a 2 m). (36) 

' C J 'C J 



Evaluating the integrals in the above upper bound we get 

1 f . , n m / 2 (bm) m / 2 n m / 2 (bm) m / 2 ( u b \ m/2 



and 



T 2 



i r p bm/Aa 

-j gix)dx= — J e -»^ dx 

M m ||a;|| 2 >6m 



4 W /2-l\ m / 2 

Vx{\\vf > bm), (38) 
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where v ~ A/"(0, 4a 2 I). Now, the probability Pr(||i/|| 2 > bm) can be tightly upper bounded as 

( —\ ' e - fem/8ff2 b > 4a 2 - 
Pr(||z/|| 2 >bm)<{ V 4 ^V 

b < 4a 2 , 



1- 

Pj n/a e - W m/8 ) 
1. 



Mb > 4; 
At& < 4. 



(39) 



Therefore, for large m, we can further upper bound (36) as follows: 
For < fi b < 4: 



u, \ 1/2 /4 e W2-l\ m / 2 

Pr(A/"(A c ) >£)<( — ) + I ] + Pr(||w|| 2 > a 2 m). 



fj, c 



For \i h > 4: 



Pr(,V(A,.) >L)< (^J + ( — ) -KPrdkll- > <r"'m). 



(40) 



(41) 



Hence, for sufficiently large m, there exists at least a lattice A* in the ensemble with complexity 



tail distribution satisfying both (40) and (41 ) for the corresponding normalized bias term \i h for 
all values of L = m + J2k=i Sk- 
it follows from standard typicality arguments that for any e > there exists m such that for 

all in > itlq 

Pr(H| 2 > a 2 m) < e/2. 



The first two terms in the upper bound (40) and (41 ) can be made smaller than e/2 for sufficiently 
large m, i.e., 

Pr(A/"(A*) > L) < e, m ->• oo, 

if fx c > 4e Mi, / 2_1 for all < fib < 4, and if /i c > /i&e Mil ' 4 when fib > 4, respectively. This result 
indicates that large computational complexity may be avoided at fi c below the cut-off VNR /i 
that is given by 

4 e w/2-i ) for o < H < 4; 
/ifee^/ 4 , for Hb > 4. 



Ho 



(42) 
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m 
k 



In this case, the total number of computations performed by the decoder can be approximated 
by 

^ (16vr) fc/2 [bk + a 2 m] k / 2 
Tj Rj m 4- > — - - - - (AV\ 

In order to see how the complexity is affected by the channel and the decoder parameters, 
we express the unconstrained AWGN channel by the vector model y = ^JJT c x + w, where x is 
the transmitted lattice point that is selected randomly from a lattice A c with generator matrix 
G c = QR, fi c is the VNR, and w ~ J\f(Q,I). The volume of the Voronoi cell of A c is selected 
such that the VNR at the output of the channel is fj, c . In this case we have V c = (2ne) m / 2 . As 
a result, we may express L as 

^T(A;/2 + l) /i ^ (det(jR T fejRfcfc)1/2) - 

It is clear from the above equation that as /i c — > oo, we have L — >■ m. Therefore, regardless 
the value of the bias term chosen at the decoding stage, the complexity of the decoder is 
approximately linear in the code dimension when the VNR is very large. This fact is also 
verified experimentally as will be shown in the sequel. 

To see now how the decoding complexity behaves as we increase the bias term, we select 
the VNR ji c > yu > yU b e Mb / 4 when /x 6 > 4. Then, one may upper bound the total number of 
computations performed by the decoder as 

f 2 [fj, b k + mfl 2 

k - r(*/2 + 1) ^ 2 e^/\det{Rl k R kk )y 2 ) 

It is clear from the above bound that as b increases (or as /x& — > oo), the computational complexity 

scales linearly with the code dimension m for all values of VNRs. The simulation results 

(introduced next) agree with the above analysis. 



L<m + T {16n)k/2 ^ bk + m] ,45) 



By taking a closer look at the upper bound ( |32| ) one can note that the term 

Pr(2(z) T w > ||a;|| 2 - bm) 

xeA* 

represents an upper bound to the decoding error probability ([9]) derived using the union bound. It 
is well-known that the union bound can provide a good estimate to the decoding error probability 
at high VNR (i.e., for \i c greater than the cut-off VNR /x ) (see IfTTlO . Therefore, achieving a 
good error performance for large values of b, where low decoding complexity is to be expected, 
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comes at the expense of increasing the VNR (or equivalently reducing the coding rate for the 
case of finite lattice codes). 

It is interesting to note that as b — > (the value of the bias that achieves close to lattice 
decoding performance) we have /i — > 4/e, where 4/e represents the gap between the cut-off rate 
and the capacity of the power-constraint AWGN channel ifTTll . In conclusion, lattice sequential 
decoders allow for a systematic approach for trading off performance, VNR, and complexity. 
For a fixed VNR, increasing the bias term allows to achieve low decoding complexity but at 
the expense of poor performance. In order to improve the performance without affecting the 
complexity, one need to increase the VNR fi c or equivalently to increase the lattice density, to 
recover the performance loss. 

VI. Simulation Results 

In our simulation we consider the unconstrained AWGN channel with m channel uses that 
is described by the vector model y = ^JJT c x + w, where x is the transmitted lattice point that 
is selected randomly from a lattice A c with generator matrix G c , fi c is the VNR, and w is an 
AWGN vector with elements that are independent identically distributed, zero-mean Gaussian 
random variables with unit variance. We consider the Loeliger ensemble of mod-p lattices, where 
p is a prime. First, we generate the set of all lattices given by A c = k(C + pZ m ), where k is a 
scaling coefficient chosen such that the Voronoi cell volume V c = (27re) m//2 , Z p denotes the field 
of mod-p integers, and C C Z" 1 is a linear code over Z p with generator matrix in systematic 
form [J P T ] T , where P is the parity-check matrix. In the following, we select a lattice code at 
random with m = 10 and p = 47 and fix the code for all simulation results. 

Fig. 3 and Fig. 4 show the effect of increasing the bias term on the average error probability 
for the case of fixed and variable bias values, respectively. In Fig. 3, we choose fixed bias values 
(independent of ji c ) and plot the average error probability versus the VNR fi c in dB. We also plot 
the performance of the optimal lattice decoder implemented via the sphere decoder algorithm 
ifTOll to measure the price of using the sequential decoder in terms of the performance loss. It is 
clear from the figure that increasing the bias term causes a right-shift to the sequential decoder 
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error probability curve, while maintaining the rate at which the curve decays^] particularly at 



high VNR values. This basically agrees with the derived theoretical results provided in (16) and 



( 17 ). On the other hand, if we let b to scale linearly with VNR as (1 — v5)// c , where < 5 < 1 



(see (18)), then according to the error exponent analysis, we expect that the rate of decay (slope) 
of the error probability curve would decrease]^] as we decrease 5. This is depicted in Fig. 4, which 
also agrees with the derived theoretical results. 

Finally, Fig. 5 shows the effect of increasing the bias term on the average computational 
complexity (defined as the total number of visited nodes during the search). For comparison, we 
also include in Fig. 5 the average complexity of the sphere decoder for the same lattice code. 
The average complexity is plotted versus the VNR in dB. It is clear that for all values of b the 
sequential decoder has much lower complexity compared to the lattice (sphere) decoder, espe- 
cially for low-to-moderate VNRs. The reason for the bell-like shape of the average complexity 
that occurs at low-to-moderate VNRs is due to the fact that with high-probability the received 
signal is close to the edge of the Voronoi cell. This basically requires the decoder to visit more 
nodes in the tree before decoding the message. As the VNR decreases or increases, the received 
signal becomes closer to a wrong lattice point or to the transmitted lattice point, respectively, 
which allows the decoder to decode the message without visiting many nodes. This leads to the 
very low average complexity as depicted in Fig. 5. The result also shows that as we increase 
the bias term, the average complexity significantly reduces, especially for low-to-moderate VNR 
values. As b — > oo, the number of computations becomes equal to m (the signal dimension) for 
all VNR values. This agrees with the derived theoretical results. 

In conclusion, simulation results indicate that increasing the bias term in the decoding algo- 
rithm significantly reduces the complexity but at the expense of losing performance. 

7 The asymptotic rate of decay of the error probability curve maybe defined as 

sloped lim ~ l Je P ^\ 

/j e ->oo log e fl c 

Now, for the case of fixed bias, using (JT7J we get 

slope = lim — lim — = lim — , 

H c ->oo 81og e /i c /J c ->oo 4l0g e fJ, c Mc->oo 81og e /i c 

which indicates that the slope of the error probability is the same for any finite b. 

8 In this case, the rate of decay (slope) of the error probability curve can be shown to be equal to 5[m/x c /8 log e /x c ] which 
depends on b via 5. 
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VII. Conclusion 

In this paper, we have analyzed the performance limits and the computational complexity of the 
lattice sequential decoder applied to the unconstrained AWGN channel. The error probability has 
been analyzed following the footsteps of Poltyrev by deriving the error exponent of the sequential 
decoder as a function of the VNR and the decoding parameter — the bias term. The bias term is 
responsible for the performance-complexity tradeoff achieved by the decoder. It has been shown 
(analytically and via simulation) that, if the bias term is fixed and independent of the VNR, 
then increasing the bias term causes only a right-shift to the error probability curve (occurs as a 
loss in the coding gain). However, if the bias term is scaled linearly with the VNR, the rate at 
which the error probability curve decays gets affected accordingly. It has also been shown that 
increasing the bias term significantly reduces the average number of computations required by 
the decoder to decode a message. However, the price of the complexity improvements comes 
at the expense of a loss in the performance. Hence, a fundamental trade-off exists between the 
error performance, the decoding complexity, and the VNR. 
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Fig. 3. Comparison of the lattice sequential decoder's performance for various values of (fixed) bias term. 
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Fig. 4. Comparison of the lattice sequential decoder's performance when the bias term varies with the VNR as fit, — (1 — v5)/^ c 
for several values of S. 
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Fig. 5. The average computational complexity achieved by the sequential decoder for different values of the bias term. 



